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Purpose 


Performance  as  a  variable  in  decision  risk 
assessment 


■  Define  the  relationship  of  system  availability  from 
acquisition  development  to  battlefield  consequences. 


■  Define  the  cost  risk  methodology 
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System  Performance  Operational 
Risk  Assessment  Tool  (SPORAT)  Concept 


INPUTS 


) 


ELEMENTS 

CONTRACTOR/GOV 

DATA-BASES 

••COST 

••SCHEDULE 

••PERFORMANCE 


•  COST,  SCHEDULE 
PERFORMANCE  ANALYSIS 


< 


c 


o 


(  TOOLS  ) 

SPORAT 

SIMULATION 


ELEMENT-LEVEL  SIMULATION 


(  USER  GROUPS  ) 


—  /PRO< 

svi— 1 —  \  R|< 


PROGRAM  ANALYST 
RISK  ANALYSTS 


INTEGRATED 
PRODUCT  TEAM 


MANAGERS/ 
DECISION  MAKERS 


Unrnssmea 


SPORAT  CAPABILITIES 


COST  CAN  BE  MODELED  USING  A  WIDE 
RANGE  OF  DATA,  BCE,  PMA,  CPR,  ETC. 

SCHEDULE  CAN  BE  MODELED  USING 
ARTEMIS,  OPENPLAN,  MACPROJECT 
PRO,  AND  MANY  OTHERS. 


PERFORMANCE  IS  DERIVED  FROM  TPMs, 
TRDs,  ELEMENT  SIMULATIONS  (EADSIM, 
TACSIM,  RADCAM,  etc.),  ACTUAL  TEST 
DATA,  EXPECTED  TEST  DATA,  OR 
THEORETICAL  ANALYSIS. 

THE  PROBLEM  CAN  BE  MODELED  AT 
ANY  LEVEL  FROM  COMPONENT  TO 
SYSTEM  OR  A  COMBINATION  OF 
VARIOUS  LEVELS. 


(  PRODUCTS  ) 


•  COST,  SCHEDULE 
PERFORMANCE 
BASELINES 

•  TRACKING  SYSTEM 

•  RISK  FORECASTS 

•  IMPACTS  OF  CHANGES 

•  WHATIF’s 


COST ;  SCHEDULE  AND  PERFORMANCE 

CAN  BE  REPRESENTED  AS 
MATHEMATICAL  FORMULAS  OR 
PROBABILITY  DISTRIBUTIONS  OR  A 
COMBINATION  OF  BOTH. 

THE  NETWORK  LOGIC  IS  FLEXIBLE 
ENOUGH  TO  MODEL  COMPLEX 
INTERRELATIONSHIPS  SUCH  AS  TEST 
FAILURES,  REDESIGN  AND  RETEST  OF 
COMPONENTS  OR  SYSTEMS,  THUS, 
PROVIDING  THE  PROBABLE  TIME,  COST 
AND  SYSTEM  PERFORMANCE 
ASSOCIATED  WITH  SUCH  A  FAILURE. 
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Network  Development 


Logic  network  of  project 

Initial  development  by 
benchmark 


Decomposition  of 
benchmark  activities 


Assignment  of  activity 
durations 


IPT  quality  review 

Input  logic  activities  into 
SPORAT 

IPT  final  verification  and 
validation 
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Overall  TPP  Matrix 


TECHNICAL 

PERFORMANCE 

PARAMETER 

APPLIED 

CONDITIONS 

FOR  DRAW 

CONSEQUENCES  IF  A 
FAILURE  OCCURS 

TYPE  HARDWARE  SOFTWARE 

TRACK  FILE  OBJECTS 
(NUMBER) 

BM  0.5  N311-N370 

BM  1.0  N591-N600 

BM  2.0  N730-N731 

NORMAL 

DISTRIBUTION 

MAX  200,  MIN  1 

BM  0.5 -MEAN  100, 

3  SEC 

STD  10 

BM  1.0 -MEAN  100, 

3  SEC 

STD  10 

BM  2.0 -MEAN  100, 

3  SEC 

STD  5 

REWORK  TIME  &  COST 

5%-1 0%  DIST. 

REWORK  DIST. 

35%-50% 

TRACK  UPDATE  LATENCY 
(SECONDS) 

BM  0.5  N465-N490 

BM  1.0  N601-N602 

BM  2.0  N740-N741 

REWORK  TIME  &  COST 

10%-20%  DIST. 

15%  HW  COST 

HDWE  INSTL  TIME 

TRACK  PURGE 
(SECONDS) 

BM  1.0  N604-N605 

REWORK  TIME  &  COST 

35%-50% 

MESSAGE  HANDLING  RATE 
(MSGS  PER  SEC) 

BM  1.0  N652-N660 

GAMMA  DIST. 

R=2 

MAX  50,  MIN  5,  1  SEC 
MEAN  17 

STD  12 

REWORK  15%  HW  COST  TIME  &  COST 

5%-1 0%  DIST. 

MESSAGE  HANDLING 
LATENCY  (SECONDS) 

BM  1.0  N654-N655 

REWORK  TIME  &  COST 

10%-20%  DIST. 

C2  PROCESSING 

NORMAL  DIST. 

MAX  200,  MIN  100 
MEAN  100 

STD  30 

SOFTWARE  REWORK  NOT  LESS  THAN  ONE  DAY 

HARDWARE  UPGRADE  INCLUDES  COST  PLUS  ONE  WEEK  DELIVERY  DELAY 
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How  TPP’s  are  Applied 


•  Tracked  Objects 
Database 

•  Incoming  Sensor 
Data  Sets 

•  Designation  of 
Objects  to  be 
Updated 


UPDATE 

OBJECT 

ASSIGNMENTS 

•  Accuracy  of  Correlations 

•  Number  of  H  its  Needed  for  Correlation 

•  Correlation  Cycle  Timeline 

•  False  Correlations 

•  Correctness  of  Correlation  Thresholds 

•  Robustness  in  Utilization  of  Inaccurate  or 
Incomplete  Sensor  Data 

•  Erroneous  Object  Updates 

•  Are  Correlation  Thresholds  Being  Met? 

•  Task  Processing  Timelines 


•  New  Object -Data 
Association 


Technical  Performance  Parameter 

Metric 

•  Accuracy  of  Object  Correlations 

•  Percent  of  Correct  Correlations 

•  Number  of  Hits  Needed  for  Correlation 

•  Average  Counts  of  Hits  (#) 

•  Correlation  Cycle  Timeline 

•  Average  Time 
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Application  of  TPPs 


(2) 


Normal  Distribution 


TRACK  FILE  OBJECTS  (BM  0.5/1 .0/2.0) 


Logarithmic  Function 


TRACK  LATEN  CY  (BM  0.5  / 1 .0  /  2.0) 


Unclassified 
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Cost 

Time 

0.5 

5-10% 

5-10% 

1.0 

35-50% 

35-50% 

2.0 

Fix:  Uniform  Distribution 


Cost 

Time 

0.5 

5-10% 

5-10% 

1.0 

H/W 

7  Days 

2.0 

Fix:  Uniform  Distribution 


Software  Integration  Phase  5 


Schedule 


ID 

Task  Name 

Duration 

4Q96  1  1Q97  1  2Q97 

[  3Q97 

Aug  |  Sep  |  Oct  |  Nov  |  Dec  |  Jan  |  Feb  |  Mar 

|  Apr  |  May 

1 

Capability  Increment  2  (CI-2) 

2 

Site  BM/C3  Design  &  Engineering 

96d 

3 

CI-2  Site  Requirements  Plan 

37cT 

4 

Site  Requirements  Maintenance 

59cT 

5 

Site  IF  Design  Described  in  IDD 

37cT 

6 

Site  IDD  Maintenance 

59cT 

7 

CI-2  IR 

Od 

10/1 

8 

CI-2  Software  Design 

44d 

9 

Software  Design 

44d 

10 

1st  Draft  DWT  Package  Complete 

Od 

*4°f1 

11 

CI-2  DWTs  Complete 

Od 

12 

Software  Implementation 

217d 

-I 

3 _ 

13 

Software  Implementation  Phase  1 

4d 

14 

Software  Implementation  Phase  II 

31cT 

Performance 


» _ 4  I 

Planning  Coding  Testing 
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Tracked  Objects  Technical 
Performance 


Analysis  Stegs 

■  Developed  Baseline  Network 

■  Strategically  Placed  Technical 
Performance  Distributions  at 
Appropriate  Benchmark  Activities 

■  Developed  and  Analyzed  Pass/Fail 
Performance  Criteria 

■  Accumulated  Progressive  Technical 
Performance  With  Narrowed 
Distributions 

■  Assessed  Cost/Schedule/Technical 
Performance 

■  Produced  Graphical  Results  for 
Cumulative  Risk  Functions 


PROJECT  COST 


$1,390 

$1,380 

$1,370 

$1,360 

$1,350 

$1,340 

$1,330 

$1,320 


Baseline 


TPMs 


Case 
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Tpp-RANGE  (NM1) 


3D  Dynamics  of  Cost,  Schedule 
&  Performance  Risk 
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Approach  to  Software  Development  Program 


For  Each  Increment  Build: 


Req  Def. 

\ 

Gov  works  w /  Cont  to: 

•  Define  Inc  Objectives 

•  Trace  to  TPM’s/  SE  Req 

•  Define  Pre-demo  Chk  List 

•  Verify  Executability 

•  Understand: 

••  Objectives 
••  Tasks 

••  Resources  Req’d 

Assess  Cont  Plan 
“Can  it  work  ?’ 


Design  &  Code  Int  &  Test  Pre-Demo 

\ _  \ _ 


Demo 

/ 


Gov  : 

•  Understands  tech 
details 

•  Guide  issue 
resolution 

•  Doc  decisions 

•  Track  issue 
resolution 


Gov 


McCabe  Tool 
(or  equiv) 

Witness  key 
testing 


Key  to  Success ... 
realistic ...  actuals 
...  DTC  focus !! 


Pre-demo  Test : 

•  Informal 

•  Verity  Chk  List 

•  Eng  Lvl 


Demo: 

•  Cont  &  Gov  share  in 
tech  briefing 

•  Show  traceability: 
TPMs /  SE  Req  to  Chk 
List,  Demo 

•  “Score  Card” 

•  Development 
Performance: 

••  S/W  Metrics 

••  Defects  Removed 
••  S/W  Rel,  Maint. 

••  Issue  Resolution 


Start  S/W  Metrics 
Foundation  . . . 


. . .  Cont  to  grow  S/W  Metrics  Data  Base 


...  Goal:  Calibrate  our  IPT  to  support 
Inc  Build  Executability  Verification 


Wkly 

Wkly 

Wkly 

Status 

Status 

Status 

■ 


Wkly 

Status 
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TP  P/TPM  Integrated 
Assessment  Capabilities 


■  Technical  Design  Minimums 

-  Design  goal  achieved 

-  What  resources 

-  Probability  of  goal 

-  Cost  impact 


■  Track  Purge  Applied 

-  Resources  need 

-  Could  the  goal  have  been  met 

-  Better  probability  of  success 

■  Deobligation  of  Funds 

-  Least  affected 

-  Completion  Affected 

-  Achievement  of  next  event 


v  Windfall  Funds 

-  Acceleration  of  tasks 

-  New  technical  goals 

v  New  Requirement 

-  Evaluate  costs 

-  Impact  on  original  schedule 

v  Network  Path  Analysis 

-  Time  slices 

-  Probability  of  success 

v  Technical  Goal  Not 
Achievable 

-  What  resources  needed 

-  What-ifs 
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System  Operational  Availability 


Unclassified 
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Comm; 

Control 


Battlefield 


Technical  Risk  &  Logistics  Analysis 


Position 

Navigation 


Assured  Communication 


:  5* 


PF  UJJ.JCWI 

*0 

Wkmi  * 


Precision 

Sustainment 


Visualization 


Combat  ID 


N 


Battlefield 

Simulation 


Logistics 


•  Performance  Risk  Identification  /Mitigation 

•  System  Maintenance  Concept 

•  Weapon  System  Concept  Trades 

•  Sensor  vs.  Interceptor  Trades 

•  Supportability  Design  Criteria 

•  Cost 

Life  Cycle  Cost /  Contractor 


Program  Cost  Schedule 
Performance 
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Contractor  Schedule 


ID 

Task  Name 

|  Duration 

1 

Capability  Increment  2  (CI-2) 

405c 

2 

Site  BM/C3  Design  &  Engineering 

96d 

3 

CI-2  Site  Requirements  Plan 

37d 

4 

Site  Requirements  Maintenance 

59d 

5 

Site  IF  Design  Described  in  IDD 

37d 

6 

Site  IDD  Maintenance 

59C 

7 

CI-2  IR 

0d 

8 

CI-2  Software  Design 

44d 

9 

Software  Design 

44C 

10 

1st  Draft  DWT  Package  Comple 

Od 

11 

CI-2  DWTs  Complete 

Od 

12 

Software  Implementation 

21 7d 

13 

Software  Implementation  Phase 

4d 

14 

Software  Implementation  Phase 

31d 

4Q96  |  1Q97  |  2Q97 


Aug  |  Sep|  Oct  |  Nov|  Dec|  Jan|  Feb|  Mar 


Apr  |  May 


~ 


Component 
Failure  Rates 


Submission  Date 

Contract  Funds  Status  Report 

accounting  quarter 

Quarterly 

A0C2 

Cost  Performance  Report  (CPR)  -  No  Criteria 

Monthly 

A003 

Cost  Performance  Report  (CPR) 

Monthly 

A005 

Date  Accession  Ust/lntemal  Data 

^S^rd 

Quarterly 

N/A 

Aqq? 

BMD  RevieWWorking  Group  Agendas 

each  PMR 

Bi-monthly 

A010 

TMD  Interoperability/Integration  Roadmap 

Deleted 

N/A^ 

N/A  V 

Ipr 

A012 

TMD  Integrated  Test  Design  &  Assessment  Plan 

3»96(30days^ 

aulSwofADII 

45  days 

i  i 


•  Failure  Distribution 

•  Filter  Logic  w/Consequences 

•  1000  Iterations 
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Component 
Failure  Rate 


Radar  Operational  Availability  Battlefield 

Environment 


37%  decrease  in  total  LRU  failure  rate 

9%  decrease  in  Sustainment  Cost 
12%  decrease  in  Leakers 


Component  Cost 


Sustainment  Simulation 


System  Operational 
Availability 

System  Sustainment  Cost 


Battlefield  Simulation 


V 


V 


> 
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